A Software Design Model for Parallel Applications on Heterogeneous Systems

Overlap of computations and communications is an effective mechanism to improve the performance of parallel/distributed applications significantly. This overlap can be achieved efficiently by using data partitioning and properly scheduling the data transfer. Various asynchronous communication primitives, that are provided by most message passing tools (e.g. PVM, MPI), can be used to implement the required overlapping mechanism. In this paper, we present a design model, the Distributed Software Design Model (DSDM) and show how it can be applied to optimize parallel/distributed applications. We show through several examples, the Master-Slave Merge Sorting Application and the astrophysical N-Body Problem, how the DSDM can be used to develop efficient and optimized implementations of parallel and distributed algorithms.